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ABSTRACT 

Reacting to potential on-orbit collision risk in an operational environment requires timely and accurate 
communication and exchange of data, information, and analysis to ensure informed decision-making for safety of 
flight and responsible use of the shared space environment. To accomplish this mission, it is imperative that all 
stakeholders effectively manage resources: devoting necessary and potentially intensive resource commitment to 
responding to high-risk conjunction events and preventing unnecessary expenditure of resources on events of low 
collision risk. After 10 years of operational experience, the NASA Robotic Conjunction Assessment Risk Analysis 
(CARA) is modifying its Concept of Operations (CONOPS) to ensure this alignment of collision risk and resource 
management. This evolution manifests itself in the approach to characterizing, reporting, and refining of collision 
risk. Implementation of this updated CONOPS is expected to have a demonstrated improvement on the efficacy of 
JSpOC, CARA, and owner/operator resources. 

1. INTRODUCTION & MOTIVATION 

On-orbit collisions pose a significant risk to satellites operating in the space environment. Recognizing the like- 
lihood and consequence of on-orbit collisions, NASA has taken several proactive measures to mitigate the risk of 
both a catastrophic loss of mission and of the increase in the space debris population. In fall 2004, NASA Goddard 
Space Flight Center (GSFC) established a process and service for identifying and reacting to predicted close ap- 
proaches for certain high-value unmanned missions based on the existing process for human spaceflight at NASA’s 
Johnson Space Center. The team responsible for executing this mission is the NASA Robotic Conjunction Assess- 
ment Risk Analysis (CARA) team. By fall 2005, this process had resulted in the execution of the first collision 
avoidance maneuver by a NASA unmanned satellite. In February 2008, NASA adopted a policy, documented in 
NASA Procedural Requirement 8715.6a - Process for Limiting Orbital Debris, that directed all maneuverable satel- 
lites to have such an on-orbit collision mitigation process. In 2009, NASA decided to require support for all opera- 
tional satellites. By January 2014, the CARA team had grown to support 65 missions, had processed nearly 700,000 
close approach messages from the Joint Space Operations Center (JSpOC), and had assisted mission customers with 
planning and executing over 75 collision avoidance maneuvers for unmanned satellites in LEO, GEO, and HEO 
orbital regimes. On average, 1000-1500 close approach notifications are received per day from the JSpOC, produc- 
ing approximately five analyzed High Interest Events (HIE) per week. 

The current CARA Concept of Operations (CONOPS) described by Newman [1] [2] has been operational since 
January 2005, and consists of a three-step process [3]: 

1. Generating close-approach predictions between the asset mission and other objects in USSTRATCOM’s 
High Accuracy Space Object Catalog (i.e., conjunction assessment, or CA). This function is performed by 
the Goddard-dedicated Orbital Safety Analysts (OS A) at the JSpOC. The OSAs are responsible for execut- 


ing screenings, performing manual orbit determinations (OD), and adjudicating tasking levels in support of 
the CA mission. 

2. Probabilistically assessing the collision risk posed by predicted close-approach events (i.e., conjunction as- 
sessment risk analysis, or CARA). This function is performed by the CARA analysts at the NASA GSFC. 
The CARA team is responsible for assessing and communicating collision risk to the satellite own- 
er/operators. 

3. Planning and executing any necessary risk-mitigating action (i.e., collision avoidance, or COLA). This 
function is performed by the satellite O/O, including mission management and the flight operations teams. 

This CONOPS was established when the CARA mission set consisted of only a handful of satellite systems. 

The analysis and reporting that was initially conceived was designed for a small group of operators who desired a 
great deal of insight into the background process. All the managers were also engineers who, instead of just wanting 
a recommendation for a maneuver, wished to know all the details of the analysis process. The CARA team encour- 
aged operators to learn more about how the process was executed by asking questions at any time. However, as the 
mission set grew, this hands-on approach became unwieldy, with CARA operators who needed to devote their time 
to high risk events instead being frequently asked to respond to questions about low risk events or purely academic 
questions. In addition, the number of conjunctions and high-interest events has been increasing since CARA estab- 
lishment and will likely continue to increase with the current and expected demand on the space enviromnent for 
commercial, scientific, humanitarian, and military purposes; generation of debris through the use of the space envi- 
ronment; and the continued investment in space surveillance and tracking capabilities to identify, detect, track, and 
catalog smaller and smaller objects. Fig. 1 shows the number of conjunctions events reported to owner/operators per 
month. In this figure, a conjunction is defined by what was reported to the owner/operator in the current CONOPS; 
namely, a conjunction within 0.5x5x5-km ellipsoid volume of the primary. Significant events that have affected the 
CARA mission set or the space debris environment are indicated by the dashed vertical lines. 
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Fig. 1: Evolution of the CARA mission, 2005 to present 

Significant manual effort is also required to analyze each conjunction to determine the risk. Although the prob- 
ability of collision (Pc) is a statement of risk (where risk here is proportional to likelihood only as consequence is 
assumed to be catastrophic), understanding how the Pc is computed, how it should be used, and how it may be ex- 
pected to behave is a very challenging endeavor. In an effort to provide that additional insight, CARA continued to 
develop new capabilities to offer decision-makers the best information. The workload to provide this service is 
high, so CARA continued to search for ways to improve the processing through automation, better algorithms, and 


new methodologies. Overlaid on Fig. 1 are several milestones in the evolution of the CARA effort. These mile- 
stones are listed and described in Table 1. 

Table 1: Significant milestones in the evolution of the CARA CONOPS 


Event 

Number 

Event (Year) 

Description 

1 

CONOPS Pronouncement 
[lj (2006) 

Formalized the CARA Concept of Operations and processes through 
documentation 

2 

Maneuver Trade Space [4] 
(2008) 

Developed a technique for collision avoidance ‘first-guess’ in order 
to reduce maneuver planning timelines 

3 

F-value [5] (2009) 

Developed a framework for enfolding other conjunction factors into a 
single conjunction risk parameter 

4 

Increase in OSA Support at 
the JSpOC (2009) 

Increasing mission sets required an increase in the number of OSAs 
from 2 to 4 at the JSpOC to support the GSFC mission; also added 
additional shifts and screening runs 

5 

CAS Re-Engineering (2010) 

Re-designed Conjunction Assessment System (CAS), which uses a 
Service-Oriented Architecture framework featuring 
GMSEC technology 

6 

Uncertainty-based Screening 
Volumes [6] (2011) 

Developed a technique for sizing screening volumes to capture events 
based on statistical uncertainties in a given orbital regime 

7 

Non-Gaussian Error Vol- 
umes (2012) [7] 

Performed an analysis that determined that non-Gaussian error vol- 
ume behavior does not significantly affect risk assessment conclu- 
sions for high-Pc events 

8 

Space Weather Trade Space 
(2014) 

Developed a technique for understanding an event’s sensitivity to 
atmospheric density mis-modeling, typically given rise by discrete 
solar events such as Coronal Mass Ejections (CME) 


Supporting the increasing demand and providing the specialized analysis required for collision risk assess- 
ment is becoming increasingly difficult in today’s resource-constrained environment, thus necessitating a stream- 
lined and efficient approach to analysis and decision-making. Therefore, a year-long effort was undertaken to re- 
design the CARA CONOPS to optimize collision risk assessment and resource management. This evolution is cen- 
tered on the ability to effectively and efficiently manage JSpOC, CARA, and Mission Operations resources, apply- 
ing operational and analytical efforts for conjunction events that pose significant collision risk and rapidly discard- 
ing conjunction events that do not. 


2. UPDATED CONOPS OVERVIEW 

While the overall CARA methodology is largely unaffected, this CONOPS evolution manifests itself in several 
aspects of the CARA process: data and information provided, mechanisms for communicating those data and in- 
formation, and courses of actions based on those communications. The changes affect all relevant stakeholders, 
including the CARA team at NASA GSFC, CARA-dedicated Orbital Safety Analysts at the JSpOC, and Mission 
Operations flight teams and management. In each step of the CARA process, the updated CONOPS ensures that 
necessary (whether situational or actionable) information be sent to stakeholders to facilitate an effective and effi- 
cient management of resources and appropriate protection of data. The key features of this CONOPS are shown in 
Table 2. 


Table 2: Key Features of the Updated CONOPS 


Aspect 

Current CONOPS 

Updated CONOPS 

Event Risk 
Characterization 
Definition (Described in 
Section 3) 

Manual assessment based 
on CARA heuristics 

Clearly-defined risk (Pc) thresholds 

Event Reporting 
Methodology (Described 
in Section 4) 

• Miss vector-based 

• Analysis and 
information provided 
for all events 

• Risk-based 

• Analysis and information dependent of event risk 

Event Refinement 
Approach (Described in 
Section 5) 

Manual assessment based 
on OSA heuristics 

Articulated strategy for examining object OD based on: 

• Calculation of an OD quality numerical score to rank- 
order which events to work 

• Enumeration of event flags to guide analyst attention 


Data, Information, and Analysis Provided 
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Decision-Level 

• Collision Probability 

• Conjunction Geometry 

• Tracking Data Amt & Quality 

• Event History 

• Visualizations 

• Maneuver Planning & Analysis 


Event Insight-Level 

• Collision Probability 

• Conjunction Geometry 

• Tracking Data Amt & Quality 


Awareness-Level 
• Pc, Conjunction Geometry only 
if previously RED or YELLOW 


Number of Events 


Fig. 2: Graphical relationship between event risk, number of events expected, the level of information 
required, and the commitment of stakeholder resources 

This paper describes the updated CONOPS and compares it to the current CONOPS by describing the analyses 
that have been performed to establish the risk characterization Pc thresholds, the changes to the reporting communi- 
cations processes, and the improvements to the OD quality evaluation for event refinement. Case studies are pro- 
vided as examples of the expected improvements. 

3. EVENT CHARACTERIZATION APPROACH 

The key enabler to the implementation of this CONOPS is the definition of a threshold for automated reporting 
based on the risk characterization of each conjunction. Having pre-defined thresholds allows CARA to associate 
products and services based on the event risk, including automation. A RED event is defined as a conjunction that 
poses a high collision risk to the primary asset. A YELLOW event is defined as a conjunction that has potential to 
become a RED (and thus high-risk) event. And, finally, a green event is defined as a conjunction that poses low 
collision risk and is unlikely to develop into a more serious threat. The basis of the event risk characterization is the 
risk (Pc) thresholds for these bins. These thresholds are similar to those posed by Carpenter using the Wald 
Sequential Probability Ratio Test [8], however, the approach used to define the thresholds here is empirical, based 
on historical conjunction data and operational mitigation actions taken. Two Pc thresholds are required in order to 
assign a conjunction one of three colors - RED, YELLOW, or GREEN. The upper Pc limit will define the boundary 
between YELLOW and RED (further referred to as the ‘Red Threshold’). The lower Pc limit will define the 
boundary between GREEN and YELLOW (further referred to as the ‘Green Threshold’). The goal of this analysis 
was to use historical conjunction disposition data to determine Red and Green Thresholds. 


In 2010, the CARA began assigning and recording “worktier” levels for events to try to capture in a quantitative 
way the amount of staff labor that each conjunction event was producing. Indirectly correlated to event risk, the 
possible worktier levels to be assigned to each event are given in Table 3 below. 


Table 3: CARA Worktier Definitions 


Event Worktier 

Worktier Definition 

1 

Owner/operator is contacted regarding a predicted conjunction 

2 

Owner/operator is provided with additional data in the form of a briefing or 
graphical plots 

3 

Mitigation activity, in response to a predicted conjunction, is planned; typically in 
the form of a collision avoidance or risk mitigation maneuver 

4 

The planned mitigation activity is executed in response to a predicted conjunction 


These worktier data were used to perform an empirical analysis to define the risk characterization thresholds [9]. By 
using the worktier data for events that required additional analysis (worktiers 2, 3, and 4), a Pc value can be 
identified above which conjunctions have typically been considered high risk. Fig. 3 below shows cumulative 
distribution function (CDF) information for event Pc values at different work-tier levels. The response for worktiers 
2-4 is tightly clumped, with all three worktiers showing similar CDF behavior, whereas worktier 1 shows a 
relatively different response, further supporting the use of worktiers 2-4 as the high-risk control group. Additionally, 
it can be seen that the curves for these three worktier levels (2+, 3+, and 4+) change their definitive shape in the 
neighborhood of the 20 th to 30 th percentile. Querying the CARA analysts about the 2+ worktier group, it was their 
feeling that about two-thirds of such events are truly high-risk, which corresponds to the percentile swath of a one- 
sigma spread for a Gaussian distribution. All taken together, these factors suggest the choice of a True Red 
threshold of about the 32 nd percentile on the CDF plot in Fig. 3, which corresponds to a Pc level of about 8.4E-04 
This number is thus a good working value for the True Red threshold. 
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Fig. 3: Cumulative distribution plot of event Pc values at different worktier levels 

The best way to use a True Red threshold is to calculate an Operational Red threshold — a smaller (more 
permissive) value — to use as a proxy for it. This is helpful because the Operational Red threshold can be 
determined such that most events that cross this threshold earlier in the event history will eventually cross the True 
Red threshold as the event develops. Using the Operational Red threshold as the boundary for RED events will 
introduce some false alarms, to be sure; but in the great majority of cases it will allow events that will eventually 
cross the True Red Threshold to be identified earlier, allowing more time to analyze and mitigate them. Using the 
historical CARA database and constraining the maximum observed Type 2 Error Rate at or below 1% for missed 
detections, the Operational Threshold to identify conjunctions which will eventually cross the True Red Threshold 
can be determined. In plain language, a boundary value below the True Red value was found for which 99 out of 100 
events whose Pc values cross that lower boundary will eventually during the development of the event cross the 
True Red threshold. This lower boundary value — the Operational Red Threshold — was calculated to be 4.4E-04. 

Applying the identified True Red Threshold from above and employing a similar Type 2 Error analysis, an 


associated Green Threshold was determined — namely, a threshold for which only a very small percentage of events 
that logged such a small Pc value would ever rise to achieve RED status. If one wishes only one out of one 
thousand events to cross from very low risk to high risk (with the latter defined as a True Red Threshold of 8.4E-04), 
this lower “Green Threshold” should be set to a value of IE-07. 

4. EVENT REPORTING APPROACH 

Having established a scheme for rapidly assigning an appropriate risk level to conjunctions, one must determine 
which conjunction-related information should be forwarded to whom and under what conditions. Under the current 
CONOPS, CARA has two primary reporting mechanisms: the routine Summary Report and a non-routine High 
Interest Event (HIE) report (or briefing); while this reporting approach is effective and should remain largely in 
place under the updated CONOPS, report details and reporting conditions do require some modification to take 
advantage of the new threshold capability. 

As previously mentioned, the data and analysis provided in a Summary Report provides insight-level 
information on the conjunction event, meaning the owner/operator will receive sufficient data and information to 
have insight into the conjunction specifics as shown in Table 4. A Summary Report is generated with each new 
prediction received. Under the Current CONOPS, it contains data for all predicted events within a 7-day prediction 
span. Under the new CONOPS, the Summary Report only includes data for previously identified or new RED or 
YELLOW Events. The Summary Report was originally developed to provide situational awareness of any predicted 
conjunctions. Within the new CONOPS, this Summary Report has evolved to provide insight into those 
conjunctions of concern; specifically, RED and YELLOW Events. Although each primary object in LEO is 
screened for seven days into the future, RED and YELLOW Events are reported in the Summary Report only within 
5.5 days of predicted TCA. This intentional lag between screening and reporting durations allows the CARA 
process to “kick in;” meaning, allowing conjunctions to be identified, characterized, and refined before unnecessary 
expenditure of owner/operator resources. RED or YELLOW Events reported at the TCA - 5.5 day point are more 
credible and can be acted on once reported, based on collision risk severity. GREEN events are reported only if they 
were previously characterized as RED or YELLOW, to allow the O/O insight into the evolution of that event. The 
advantages of such a reporting schema are that owners/operators are not presented with excessive, non-actionable 
information, and CARA analysts are not subjected to questioning about events that are not high-risk and in most 
cases unlikely to become so. 

The second major CARA product, the HIE briefing, provides the decision-level information for events requiring 
mitigation planning and will still consist of the full suite of CARA analysis capabilities and products. The contents 
of the HIE briefing are largely unchanged in the updated CONOPS, but the delivery criteria are different. Under the 
current CONOPS, the HIE package is delivered for events that the CARA team heuristically determine to be high 
threat based on various factors, while under the updated CONOPS, they will be provided for all RED events. The 
HIE briefing contains all the information in the Summary Report as well as additional information as shown in Table 
4. RED events are those events deemed to be a current high risk and therefore should receive all information and 
analysis products to aid in maneuver planning and commitment. In addition to the HIE briefing product, a CARA 
analyst will also support meetings and brief the package so that any owner/operator questions can be answered in 
person. 


Table 4: Standard CARA Products 


CARA 

Product 

Data, Information, and Analysis Contained 

Current CONOPS 

Updated CONOPS 

Content 

Delivery Criteria 

Content 

Delivery Criteria 

Summary 

Report 

• Current Pc, Pc history 

• Current Miss Vector, 
Miss Vector history 

• TCA<7daysfor 
LEO, TCA < 10 
days for GEO, 
HEO 

• Miss vector 
within 0. 5x5x5- 
km box for LEO, 
miss distance 
within 15 km for 
GEO, HEO 

• Current Risk, Risk 
history 

• Current Pc, Pc history 

• Current Miss Vector, 
Miss Distance history 

• Relative geometry 
visualization 

• Current miss vector 
uncertainty 

• Secondary object 
tracking information 

• TCA< 5.5 days 
for LEO 

• Any RED or 
YELLOW events 

HIE Briefing 

• Same information as 
Summary Report, 
plus 

• Miss Vector 
uncertainty history 

• Secondary object 
information 

• Relative geometry 
and relative motion 
visualizations 

• Primary and 
secondary object 
positional 
uncertainty history 

• Miss Sigma Level 
plot (OCM 
consistency check) 

• Space Weather trade 
space 

• Maneuver planning 
trade space 

• Maneuver screening 
results 

Any event deemed 
high-interested by 
the CARA team 

No Change 

All RED Events 


Table 5 below summarizes the actions of each node of activity (JSpOC, CARA, and O/O) that occurs upon 
receipt of new data from the JSpOC based on the CONOPS-related color assignment of the event as listed in the 
Summary Report. 


Table 5: CA-Related Activities at Each Process Node Based on Conjunction Risk Level 


Event 

Type 

OSA Actions 

CARA Actions 

Owner / Operator 
Actions 

RED 

Event 

1. Examine and adjust the orbit 
determination solution as 
necessary for every offending 
secondary object 

2. Adjudicate the tasking level of 
every offending secondary 
object 

1 . Perform risk analysis 

2. Generate and deliver HIE package 

3. Support HIE briefings as necessary 

4. Assist with mitigation planning 
activities 

5. Coordinate maneuver screenings with 
JSpOC and evaluate maneuver plans 

1 . Receive routine 
Summary Report 

2. Receive HIE 
briefing from 
CARA 

3. Determine 
mitigation options 

4. Plan mitigation 
activity as 
necessary 

5. Execute mitigation 

YELLOW 

Event 

1. Examine and adjust the orbit 
determination solution as 
necessary for offending 
secondary object in priority 
order 

2. Adjudicate the tasking level 
of offending secondary object 
in priority order 

1. Perform risk analysis 

2. Evaluate whether adjustment to red 
status is necessary 

Receive routine 
Summary Report 

GREEN 

Event 

No action required 

No action required 

No action required 


4. EVENT REFINEMENT APPROACH 

The CARA process starts with the conjunction identification, or screening, at the JSpOC. For LEO assets, this 
prediction period is 7 days; for GEO/HEO, the prediction period is 10 days. Screening out for multiple days enables 
identified conjunctions to be worked both by the OSA team at the JSpOC and CARA team at GSFC to refine and 
improve the orbit knowledge of the objects involved. This refinement hopefully improves the ability to determine 
whether the conjunction is a high risk, making mitigation action prudent, or a low risk, and thus can be disregarded 
as posing a collision risk. This refinement of orbit knowledge is typically accomplished by performing a manual 
orbit determination (OD), allowing improvements such as adjusting the fit span or removing “bad” observations, or 
attempting to collect more observations on the secondary object through increasing the tasking priority to the Space 
Surveillance Network (SSN). Under the current CONOPS, the OSAs work conjunctions events that violate a 
specified volume, which does take cognizance of the OD quality but not of the event risk. The current refinement 
process does not consider either risk or quality in a structured manner. This could result in unnecessary requests for 
increased tasking, further burdening the SSN. With the updated CONOPS, there is a rigorous and robust strategy for 
event refinement. Under the updated CONOPS, all conjunctions will be characterized by their inherent collision 
risk to the primary asset (i.e. risk color); and this characterization drives what data, information, and analysis are 
conducted and delivered for which conjunction events and when. In this CONOPS, there is a second hierarchal 
organization of conjunction events — a conjunction “work list.” This work list prioritizes the conjunctions events to 
be worked and includes two aspects of the conjunction: the apparent collision risk as described by the collision 
probability and the quality of the prediction and input data into the Pc computation. This combination of risk and 
quality is similar to the approach described by Frigm in the implementation of the F-value construct [5]. The 
worklist organizes events according to the three-stage strategy below: 

1 . All events are grouped by event risk (RED events first, YELLOW events second) 

2. All events are ordered within each group by OD quality (poorest quality to highest quality) 

3. All events are flagged for any unique characteristics, if applicable 

The grouping is autonomously performed using the risk color methodology previously discussed. A numerical 

score will now be used to autonomously rank the events into this worklist for the OSAs. This work list helps focus 
OSA and JSpOC resources on those events most needing attention based on CA risk assessment, not on miss 
distance or amount of tracking, as done in the current CONOPS. This change enables tracking resources to be 
requested only on those conjunctions that are actually threats. The work list first groups events by the risk 
characterization schema defined previously and the rank-order within each group. The OSAs will perform their 


responsibilities in accordance with the prioritization of events within the work list, meaning the OSAs will first 
perform manual ODs and tasking adjudication for the RED events and then proceed with the YELLOW events in 
rank-ordered fashion as resources permit. The event flags are used to provide guidance on specific aspects on which 
the OSA should focus their attention when the event arrives in the queue. This prioritization and flagging allows for 
the most severe events or events whose Pc is calculated off of the poorest quality and therefore has the potential of 
becoming high risk with additional tracking or orbit refinement to be attended to first. 

In order to perform this rank-ordering, the ‘scoring’ of the quality of the input data is performed via a 
methodology that considers both the solution quality resulting from the estimation process and the quality resulting 
from the propagation of that state and state uncertainty. Since the primary object is often tracked very well and its 
orbit very well known (with the exception of solutions shortly after a maneuver), this scoring is performed on the 
offending secondary object. Three key aspects of OD quality were analyzed and a methodology developed to 
compute a numeric score based on the evaluation of those aspects: the tracking adequacy (that is, the amount 
tracking used in the estimation process), the distribution of tracking about the object’s orbit, and OD residual 
analysis. 

Tracking Adequacy 

The evaluation of the adequacy of the amount of tracking data in an OD is not a straightforward investigation. 
The very general principal of “more tracking tends to produce a better OD” is clear enough; but what is the precise 
trade-off between tracking density and OD quality? Is there a point of diminishing return beyond which the 
marginal improvement in OD quality with each additional track is essentially negligible? How much does this 
“adequacy function” change for satellites in different orbit regimes? 

Fortunately, a large study has recently been conducted that is very helpful in answering these questions, or at 
least providing a framework for such answers. The Space Surveillance Network Optimization Study, Issue II 
(abbreviated SSNOII) [10], performed by AFSPC/SMC and NASA/GSFC, developed a set of functional 
relationships between OD prediction error and object tracking density, with experimental controls for satellite orbit 
regime, the different types of sensors contributing to the tracking mix, the propagation interval, and the statistical 
confidence level of the OD error. These relationships are expressed as piecewise-continuous curves that give the 
relationship between OD vector prediction error and the number of sensor tracks in the OD interval. Figure 4 below 
gives an example of one such curve, which is illustrative of the general behavior of the entire curve-set. It will be 
noticed that this curve manifests rapidly-changing behavior at the lower levels of tracking but eventually reaches a 
region where there is rather little marginal change in OD error with increased tracking density; this region is called 
the “steady-state” area and can be approximated with a single value for accuracy (as it is essentially a horizontal line 
in comparison to the other portions of the plot). In Fig. 4, an estimate of the steady-state value is given by the 
dashed magenta line. 



Fig. 4: SSNOII example curve (estimates of steady-state levels given in magenta, actual tracking level for a 

particular example OD given by the asterisk.) 

While these plots were developed to give actual OD error estimates for specified tracking densities, a secondary 
use is to calculate a factor that characterizes the adequacy of the OD tracking data. The accuracy level that 
constitutes the steady-state response region (Vss) can be used as a standard of sorts; tracking levels that produce this 


level of OD error represent the region of best response and are thus considered fully adequate. For tracking levels 
lower than this, the ratio between the OD error at that tracking level (Vobj) and the steady-state OD error (Vss) can 
serve as a characterization factor for the tracking adequacy. This ratio is an attractive candidate for such a factor 
because it formulates the tracking level adequacy into a single parameter with reasonably-determined boundaries 
(lower-bounded at 1, upper bounded at ~50) and controls for the different thresholds of adequacy required for 
different orbit regimes (because the SSNO II curves differ by orbit regime). This ratio is floor-limited at unity, as 
tracking beyond a certain point provides so little relative marginal improvement that it can effectively be considered 
negligible. A formal statement of the parameter calculation is as follows: 


A r 



A r 


A r , A r > 1 
.1, A r < 1 


( 1 ) 


Tracking Distribution about the Orbit 

The distribution of tracks about the orbit arc does indeed affect the quality of the resultant OD, but the proper 
way to characterize this phenomenon is not immediately clear. Similar to the effect that was observed with the 
SSNO II curves, past a certain level of distribution about the arc any increasing salutary effect becomes minor; this 
point was estimated to be about 50% of the orbit arc. Rather than try to calculate a continuous function, such as the 
maximum angular separation (in true anomaly) between tracks, it was considered preferable to divide the orbit arc 
into some number of equal arc segments (again in terms of true anomaly) and tabulate instead the percentage of 
these divisions that contained tracking data. Typically half of the divisions populated with tracking data would 
represent essentially full orbit distribution coverage, with fewer indicating a less sanguine situation. The actual 
factor - the orbit coverage statistic (OCS) - is determined by the equation below (in which n represents the number 
of divisions); one can imagine different formulations, but the below conforms to general expectations: it presumes 
that the situation is twice as unfavorable if only one sector of a six-sector-division orbit is populated. 

OCS = (^—) x + f + i^ 2 -) (2) 

Vl-n/2/ Vl-n/2/ Vn/2 — 1/ v ' 

Finally, it seemed appropriate to add an additional penalty if the sector in which the actual close approach would 
occur did not contain any tracking data. The final functional form for a six-sector division of the orbit (the nominal 
number of divisions expected to be used at the initial operational roll-out) is the following: 

OCS — 2.5 — O.Sx + .25 (penalty if TCA not covered ) (3) 

One can see that the range of values for the orbit distribution factor spans from a minimum of unity to a maximum 
of 2.25 (only one sector sampled, and that sector does not contain the time of closest approach). Fig. 5 below is an 
example of the kind of display that might be shown to an owner/operator to illustrate the tracking distribution 
situation, with the darker areas showing sectors that contain tracking and the magenta asterisk indicating the 
satellite’s position at TCA. Notice that, because the orbit is somewhat eccentric, the sectors are not of equal arc 
length, as they are shorter at perigee due to the more rapidly changing orbit in that area; this is similar to the 
phenomenon of using Cowell regularization in propagation to shorten the step-size of HEO orbits near perigee. 



Fig. 5. Orbit distribution plot for a slightly eccentric orbit. Two of the six orbit sectors are populated, 
yielding a factor > 1. The magenta asterisk indicates the sector in which TCA will occur. 

OD Residual Analysis 

The remaining factor to profile is that for parameters that speak directly to the quality of the OD: the weighted 
residual root-mean square (RMS) value and the percent of residuals accepted (retained) in the OD. Other OD 


quality factors could in principle be calculated and used, but these two parameters were readily available and 
appeared to be able to characterize the situation adequately. 

The weighted RMS is the square root of the average of the squares of the component residuals, each of which is 
first normalized by the variance of the observable error for that particular sensor. As such, the expected value of the 
weighted RMS is unity, and any deviation (in either direction) is a sign of a sub-optimal OD — either that the residual 
errors are much greater than expected or that the residual weights are not properly set. Since the RMS can in some 
instances be improved by eliminating tracking data in the OD, it is prudent to normalize this factor by the percent of 
residuals accepted: as this percentage drops, the overall factor increases in value, indicating a worse score. 

Composite OD Quality (ODQ) Factor 

The proposed overall composite functional form, chosen as a linear form to simplify tuning, for the omnibus 
OD quality factor is thus the following: 

ODQ = k 0 + k,Ar + k 2 ( OCS ) + k 3 ^ (4) 

While preliminary analyses indicate that the above form is quite adequate for the present purposes, as the 
relationship is further tuned to obtain values for the constants it may also be expedient to add correlation terms or 
other standard control variables. 

Event Flags 

In addition to the rank-ordering of conjunctions events described above, the work list also contains event flags. 
These flags do not alter the grouping and ranking as described in the previous section but help guide the analysts in 
manually evaluating each conjunction, and help focus analyst attention on specific event characteristics. The flags 
are arranged into two different categories, Event Driven and Object Driven, as described below and shown in Table 
6 . 

1. Event driven flags are specific to the given conjunction event. For example, if the relative velocity 
between the two objects is sufficiently low, the 'Low Relative Velocity' flag is set. This flag would indicate 
to the CARA team that further analysis should be performed, including computing the collision probability 
via a Monte Carlo simulation, and compare results to the 2-D numerically integration collision probability 
results. 

2. Object driven flags are flags that are specific to either the primary or secondary object. One example of 
an object flag is the force model settings quality control flag. This flag would indicate that the current 
force models that are being used don't match expected, pre-defined settings. This flag would indicate to the 
CARA team that something has changed in the modeling, and to ask the JSpOC OSAs for clarification on 
why the change has occurred. 


Table 6: Example Event Flags used in the Worklist 


Flag Type 

Example 

Event 

V Large Change in relative R-I-C miss sigma level 

V Low Relative Velocity 

V O/O -ASW Pc Difference 

V Repeating Conjunction Event 

V Single Station Tracking 

Object 

V Known maneuverable spacecraft 

V Large RCS 

V Large Covariance 

V Propagation Time 


5. CASE STUDIES 

In this section, two sample case studies are presented. In each case, the event is ‘walked through’ the current 
and updated CONOPS highlighting the prominent activities by the CARA team. The data and information presented 
for each case is admittedly insufficient for a comprehensive picture of the conjunction event but is simplified here 
for purposes of illustrating the benefits of the updated CONOPS. Both cases are conjunctions which were identified 


6 days prior to TCA through the normal screening process at the JSpOC, and both received a daily updated 
prediction of that event (i.e., a new OCM with updated estimates of the states and state uncertainty of the two 
objects propagated to the TCA). 

In case study #1, whose details are shown in Table 7, the conjunction was first identified as having a miss 
distance of 235 meters and a Pc of 3.7E-04. As the event evolved, the miss distance increased to about 500-550 
meters; the Pc rolled off precipitously after the second update. In the current CONOPS, this event would have 
required significant effort and interaction between the CARA team and the O/O. Both the miss distance and Pc were 
in ranges that the O/O typically consider worthy of investigation and pursuit. Thus, maneuver planning would have 
likely occurred but eventually been waived off. In the updated CONOPS, this event never would have been 
characterized above YELLOW. When characterized as YELLOW, the event would appear in the automated CARA 
reports and would contain additional information beyond the event descriptors, Pc, and conjunction geometry; it 
would also contain information of the tracking quantity and quality. The OSAs at the JSpOC would have also 
examined the orbit determination and adjudicated the tasking for the duration of its characterization as YELLOW in 
rank-order of all YELLOW events. Since it was characterized as YELLOW at one point during the event prediction, 
it will remain on the CARA reports even after the characterization went to GREEN at TCA - 3 days. With the risk- 
based characterization and reporting, the updated CONOPS would have correctly provided insight-level information 
but no additional action would have been taken, saving valuable resource expenditure by the CARA team and the 
mission operations team. 


Table 7: Case Study #1 - “False” High-Risk Event in Current CONOPS 


Days Before 
TCA: 

6 

5 

4 

3 

2 

1 

Miss Distance 

235 m 

336 m 

452 m 

517 m 

542 m 

533 m 

Collision 
Probability (Pc): 

3.7E-04 

1.2E-04 

2.3E-06 

7.6E-09 

0 

0 

Likely Activity 
in Current 
CONOPS: 

• Report 
event to 
O/O 

• Field ini- 
tial ques- 
tions from 
O/O 

• Report 
event to 
O/O 

• Brief 
HIE 

Package 

• Report event 
to O/O 

• Brief HIE 
package 

• Support ma- 
neuver plan- 
ning 

• Report event 
to O/O 

• Brief HIE 
package 

• Evaluate 
maneuver 
plan 

• Report 
event to 
O/O 

• Brief HIE 
Package 

• Report 
event to 
O/O 

• Waive 
Maneu- 
ver 

Conjunction 
Risk in Updated 
CONOPS: 

YELLOW 

YELLOW 

YELLOW 

GREEN 

GREEN 

GREEN 

Pre-determined 
Activity in Up- 
dated CONOPS: 

Not reported 
/ no activity 

Report 
event to 
O/O in 
Summary 
Report 

Report event 
to O/O in 
Summary Re- 
port 

Report event 
to O/O in 
Summary Re- 
port 

Report event 
to O/O in 
Summary 
Report 

Report 
event to 
O/O in 
Summary 
Report 


In case study #2, whose details are shown in Table 8, the conjunction was first identified as having a miss 
distance of 6 kilometers and a Pc of 3.7E-04. As this event evolved, both the miss distance and Pc remained fairly 
consistent. In the current CONOPS, the reporting criterion is miss vector-based, a 0.5x5x5-km ellipsoid; therefore, 
this event would have never appeared on a CARA report to the O/O. In the updated CONOPS, the reporting is risk- 
based, therefore, this event would have appeared on the CARA Summary Report within 5 days of TCA. Once the 
event achieved a RED characterization at TCA - 4 days, CARA would have begun generating and briefing HIE 
packages and recommending maneuver planning. For this event, the updated CONOPS would have correctly 
characterized the event risk and prompted the necessary actions and exchange between the CARA team and the O/O. 


Table 8: Case Study #2 - “Missed” High-Risk Event in Current CONOPS 


Days Before 
TCA: 

6 

5 

4 

3 

2 

1 

Miss Distance 

6.0 km 

6.3 km 

6.2 km 

6.3 km 

6.1 km 

6.0 km 

Collision 
Probability (Pc): 

3.7E-04 

4.2E-04 

8.5E-04 

1.7E-03 

1.5E-03 

2.0E-03 

Likely Activity 
in Current 
CONOPS: 

No report / 
no activity 

No report / 
no activity 

No report / no 
activity 

No report / no 
activity 

No report / no 
activity 

No report / no 
activity 

Conjunction 
Risk in Updated 
CONOPS: 

YELLOW 

YELLOW 

RED 

RED 

RED 

RED 

Pre-determined 
Activity in Up- 
dated CONOPS: 

No activity 

Report 
event to 
O/O in 
Summary 
Report 

• Report event 
to 0/0 in 
Summary 
Report 

• Brief HIE 
package 

• Support ma- 
neuver plan- 
ning 

• Report event 
to 0/0 in 
Summary 
Report 

• Brief HIE 
package 

• Evaluate 
maneuver 
plan 

• Report 
event to 
0/0 in 
Summary 
Report 

• Maneuver 
Commit 

• Report 
event to 
0/0 in 
Summary 
Report 

• Execute 
Maneuver 


6. CONCLUSIONS 

This paper provides an overview of the updates to the CONOPS being deployed at NASA GSFC for the NASA 
Robotic Conjunction Assessment Risk Analysis effort. The updated CONOPS focuses on providing the necessary 
data, information, analysis, and support to ensure accurate, timely, and efficient management of collision risk and 
operational resources. The updated CONOPS manifests itself through a risk-based characterization and reporting 
strategy and a risk and quality-based approach to event prioritization for refinement, enabling an efficient and 
effective resource management that is commensurate with event severity or risk. The key features of the updated 
CONOPS are shown in Table 9. 

Table 9: Key Features of the updated CARA CONOPS 


Aspect 

Feature 

Benefit 

Event Risk Characterization 
Definition 

Clearly-defined risk (Pc) thresholds 

• Improved automation 

• Standardize service 

• Level set expectations 

Event Reporting 
Methodology 

• Risk-based 

• Analysis and information dependent of 
event risk 

• No resource expenditure on low 
risk events 

• Owner/operator receives more 
information and more frequently 
more high risk events 

Event Refinement Approach 

Articulated strategy for examining object 
OD based on: 

• Calculation of an OD quality numerical 
score 

• Enumeration of event flags 

• Object tasking for high risk events 
only, not all events 

• Manual activities prioritized by 
OD quality numerical scoring 


Updates to operational procedures and software systems are currently under development to implement this 
CONOPS. The CARA process provides a conjunction risk analysis service to over 65 robotic on-orbit satellite 
systems. The CARA Team feels that this CONOPS is the next evolution in enhancing the service it provides. 


7. FUTURE WORK 


The updated CARA CONOPS will continue to evolve as the body of knowledge in the conjunction assessment 
community increases and advanced technologies are incorporated. Through the development and implementation of 
the updated CONOPS, the authors have identified several areas for continued investigation: 

1 . Development of additional solution and prediction quality assessments, and integration into the quality 
scoring algorithm 

2. Development of additional flags as unique conjunction aspects are observed in CARA operations 

3. Re-factorization of the F-value to incorporate this quality assessment algorithm 

4. Periodic re-evaluation of the Red and Green Thresholds 

5. Examination of covariance-based screening at the JSpOC to enable a closed-loop probabilistic risk 
assessment paradigm 
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